Voice Activity Detection for Ultrasound-Based Silent Speech Interfaces Using Convolutional Neural Networks

Voice Activity Detection (VAD) is not easy task when the input audio signal noisy, and it even more complicated an recording. This case with Silent Speech Interfaces (SSI) where we record movement of articulatory organs during speech, aim to reconstruct speech from this Our SSI system synthesizes ultrasonic videos tongue movement, quality resulting signals are evaluated by metrics such as mean squared error loss function underlying neural network Mel-Cepstral Distortion (MCD) reconstructed compared original. Here, first demonstrate that amount silence in training data can have influence both on MCD evaluation metric performance model. Then, train a convolutional classifier separate silent speech-containing ultrasound images, using conventional VAD algorithm create labels corresponding signal. In experiments our ultrasound-based speech/silence separator achieved classification accuracy about 85\% AUC score around 86\%.